EQUIVIO at TREC 2009 Legal Interactive

Equivio participated in two runs under the legal interactive track: topics 205 and 207. The runs utilized the Equivio>Relevance product. Equivio>Relevance is an expert-guided system which enables automated prioritization of documents and keywords. Based on initial input from a lead attorney, Equivio>Relevance uses statistical and self-learning techniques to calculate graduated relevance scores for each document in the data collection. Equivio>Relevance works like this: • The software generates a sample of documents, and feeds them to an expert, that is, an attorney who is familiar with the case. • The expert tags the sample documents as relevant or not. • These samples are used to train the software. Based on the expert's decisions for the sample documents, the software learns how to score documents for relevance. • In an iterative, self-correcting process, Equivio>Relevance feeds additional samples to the expert. These statistically generated samples allow Equivio>Relevance to progressively improve the accuracy of its relevance scoring. • Once the sampling process has optimized, the software indicates that it has reached "stable" mode. "Stable" mode is an indication that additional samples are not required. At this point, the batch scoring can be invoked. In this process, the software calculates a graduated relevance score for each document. Each sample comprises 40 documents. Typically, 25 through 50 iterations, each of 40 documents, are required in order for the software to stabilize and optimize the sampling. Equivio>Relevance comprises three key modules – an interactive model used by the expert for tagging documents, a statistical model which serves to monitor and manage the iterative sampling process, and a text classification algorithm. The statistical model manages sample selection. The samples are fed to the expert, and the input from the samples is used to train the classifier.